Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Real-time data analysis system based on Spark Streaming and its application
HAN Dezhi, CHEN Xuguang, LEI Yuxin, DAI Yongtao, ZHANG Xiao
Journal of Computer Applications    2017, 37 (5): 1263-1269.   DOI: 10.11772/j.issn.1001-9081.2017.05.1263
Abstract1050)      PDF (1159KB)(801)       Save
In order to realize the rapid analysis of massive real-time data, a Distributed Real-time Data Analysis System (DRDAS) was designed, which resolved the collection, storage and real-time analysis for mass concurrent data. And according to the operation principle of Spark Streaming, a dynamic sampling K-means parallel algorithm was proposed, which could quickly and efficiently detect all kinds of DDoS (Distributed Denial of Service) attacks. The experimental results show that the DRDAS has good scalability, fault tolerance and real-time processing ability, and along with new K-means parallel algorithm, the DRDAS can real-time detect various DDoS attacks, and shorten the detecting time of attacks.
Reference | Related Articles | Metrics